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Abstract 


A predominant research focus in the free flight community has been 
on the type of information required on the flight deck to enable pilots to 
"autonomously" maintain separation from other aircraft. At issue are 
the relative utility and requirement for information exchange between 
aircraft regarding the current “state” andlor the “intent” of each 
aircraft. Trajectory predictions based on this information are used in the 
detection of possible losses of separation or “conflicts,” and accurate 
conflict detection capability is a key requirement for autonomous aircraft 
operations. Investigation of separation assurance in constrained 
operations may lead to a system-level determination regarding the 
advisability of exchanging both state and intent information to enable the 
human participants to achieve all objectives and meet all constraints 
with long-term stability and safety. Relevant operational constraints 
include traffic flow management requirements, airspace hazards, aircraft 
performance limitations, and operational economic considerations. 

This paper presents the experimental design and some initial findings 
of an experimental research study designed to provide insight into the 
issue of intent information exchange in constrained en-route operations 
and its effect on pilot decision making and flight performance. The 
piloted simulation was conducted in the Air Traffic Operations 
Laboratory at the NASA Langley Research Center. Two operational 
modes for autonomous operations were compared under conditions of 
low and high operational complexity. The tactical mode was 
characterized primarily by the use of state information for conflict 
detection and resolution and an open-loop means for the pilot to meet 
operational constraints. The strategic mode involved the combined use 
of state and intent information, provided the pilot an additional level of 
alerting, and allowed a closed-loop approach to meeting operational 
constraints. Potential operational benefits of both modes are illustrated 
through several scenario case studies. Subjective data results are 
presented that generally indicate pilot consensus in favor of the strategic 
mode. Pilot comments presenting merits and criticisms of the 
operational modes are included, as are usability assessment ratings for 
user-interface design features. 
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Abbreviations 

ADS-B 

Automatic Dependent 
Surveillance - Broadcast 

ASTOR 

Aircraft Simulation for Traffic 
Operations Research 

ATC 

Air Traffic Control 

ATM 

Air Traffic Management 

ATOL 

Air Traffic Operations Laboratory 

ATSP 

Air Traffic Service Provider 

CDTI 

Cockpit Display of Traffic 
Information 

CDU 

Control and Display Unit 

CNS 

Communications, Navigations, 
and Surveillance 

DAG TM 

Distributed Air/Ground Traffic 
Management 

FCP 

Flight Control Panel 

FL 

Flight Level 

FMS 

Llight Management System 

MCP 

Mode Control Panel 

NAS 

National Airspace System 

NASA 

National Aeronautics and Space 
Administration 

ND 

Navigation Display 

NLR 

National Aerospace Laboratory of 
The Netherlands 

PFD 

Primary Flight Display 

PredASAS 

Predictive Airborne Separation 
Assurance System 

RTA 

Required Time of Arrival 

SUA 

Special Use Airspace 

TCP 

Trajectory Change Point 

TFM 

Traffic Flow Management 

TMX 

Traffic Manager 

Introduction 


A large portion of the aviation user 
community has identified a need for increasing 
flexibility of aircraft operations while retaining 
guaranteed separation from hazards. This need 
has been expressed as a new operational 
paradigm, “free flight,” which reduces reliance 
on centralized air traffic management. Free 
flight is defined by the RTCA Task Force 3 as a 
safe and efficient flight operating capability 
under instalment flight rules in which operators 
have the freedom to select their path and speed 


in real time 1 . One approach to achieving 
mature-state free flight is to distribute 
responsibility and capability for traffic 
management between aircraft and ground-based 
air traffic control over as much airspace as 
possible, while minimizing the mandating of 
equipage for airspace access. 

A significant research activity within the 
NASA Aviation System Capacity program is 
focused upon far-term operations of the National 
Airspace System (NAS). A general description 
of the activity is Distributed Air/Ground Traffic 
Management (DAG TM). DAG TM is based on 
the fundamental premise that all NAS 
participants can be both information suppliers 
and users, thereby enabling collaboration and/or 
distribution in all levels of traffic management 
decision making. Successful operation in this 
new environment will be achieved through new 
human-centered operational paradigms enabled 
by procedural and technological innovations. 
These innovations include decision-aiding 
automation, information sharing, and 
communication, navigation, and surveillance / 
air traffic management (CNS/ATM) 
technologies. 

In planning for the DAG TM research 
activity, NASA has developed a high-level DAG 
TM concept of operations consisting of 15 
elements spanning gate-to-gate operations 2 . One 
particular DAG TM concept element developed 
to address the en-route flight regime (Concept 
Element #5) has the potential to increase 
capacity, flexibility, and robustness of the NAS 
by distributing responsibility for 1) separation 
assurance and 2) conformance with local traffic 
flow management (TFM) constraints between 
airborne and ground-based systems, hr this 
concept element, pilots of aircraft designated as 
“autonomous” have the authority to generate and 
implement new trajectories at their discretion in 
order to meet individual-, company- (if 
applicable), and/or system-level goals. They 
also have the responsibility for separation 
assurance and compliance with local TFM 
constraints established by the ground-based air 
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traffic service provider (ATSP). Aircraft not 
operating as autonomous aircraft are designated 
as “managed aircraft,” and similar to current 
operations, their flight crews comply with 
clearances provided by the ATSP, who 
maintains responsibility for separation assurance 
and flow management conformance of these 
aircraft. 

Information Requirements Research 

A predominant research focus in the free 
flight community has been on the type of 
information required on the flight deck of 
autonomous aircraft to enable their pilots to 
ensure separation from other aircraft. Accurate 
detection of “conflicts” or predicted loss of 
separation between aircraft is a key requirement 
for autonomous aircraft operations. At issue are 
the relative utility and requirement for inter- 
aircraft information exchange of the current 
“state” (three dimensional position and velocity 
vector) and “intent” of each aircraft; this 
surveillance information fomts the basis for 
trajectory predictions used in automated conflict 
detection. Additionally, related human factors 
issues exist, such as determining how pilots 
would use the surveillance information and how 
this information should be presented on the 
flight deck displays, considering usability, 
display design precedence, and integration with 
other pilot tasks. 

Previous research has indicated that, under 
unconstrained operations (no schedule or 
airspace restrictions), the exchange of state 
information between aircraft is sufficient to 
safely enable airborne self separation in the en 
route domain 3 . A state-only system has the 
potential to significantly reduce bandwidth 
requirements for future surveillance systems 
such as Automatic Dependent Surveillance - 
Broadcast (ADS-B), and it reduces the 
complexity of conformance monitoring and 
conflict alerting logic. To address the conflict 
alerts missed by not using intent infomiation, 
Hoekstra et. al . 3 developed and tested a 
predictive airborne separation assurance system 
(PredASAS) that calculates potential off- 


trajectory conflicts by comparing the current 
state vectors of traffic aircraft to possible 
changes of the own-ship (i.e., subject aircraft 
performing self- separation) state vector. Own- 
ship maneuvers that would result in traffic 
conflicts are displayed as avoidance bands on 
the heading, airspeed, and vertical speed 
indicators. This system was designed to provide 
information regarding which maneuvers would 
lead to a conflict without the crew needing to 
"probe" or "try various maneuvers." The 
conclusion was made that "if all equipped 
aircraft are fitted with PredASAS, there is no 
longer a need to know intent information 
because nobody will turn (or climb/descend) 
into a conflict ." 3 

In contrast, a study conducted at the NASA 
Ames Research Center suggested that pilots 
nevertheless preferred to be provided traffic- 
aircraft intent infomiation, and the preferred 
source of intent infomiation was Flight 
Management System (FMS) flight plan data 4 . In 
this study, flight crews were alternately provided 
with three types of traffic infomiation: state 
data, Mode Control Panel (MCP) data, or FMS 
data. The flight crews were given the 
opportunity to use voice communication 
channels to communicate directly with other 
aircraft to gather intent infomiation or negotiate 
resolutions. Results of the study indicated that 
pilot preferences for intent infomiation centered 
primarily on the improved ability to understand 
the conflict alerts. Intent infomiation type (state, 
MCP, or FMS) was found to have no effect on 
separation assurance. 

Operational Constraints 

Little research has been perfomied on the 
feasibility of constrained operations. 
Constrained operations are important to consider 
in concept feasibility and viability analyses, 
because operational constraints ultimately limit 
airspace capacity (notwithstanding runway 
availability limitations), and a concept that does 
not address capacity limitations is of little 
practical interest. It is in the more highly 
constrained conditions that operations will be 
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found to be either fragile or robust to real-world 
system demands and variability. 

Operational constraints can generally be 
expressed in four categories: flow management, 
airspace hazard, performance, and economic. 
These four categories of constraints, together 
with constraints provided by complex traffic 
geometry and dense traffic volumes, must be 
considered in the assessment of autonomous 
operations feasibility. 

Flow management constraints are restrictions 
that must be imposed to make sure traffic flow 
through the airspace is as high as possible. 
Examples of such constraints in the current 
system are speed and altitude restrictions at fixes 
and speed restrictions en route in order to 
regulate the flow of aircraft, for example, into 
capacity-limited terminal airspace. In a future 
system, the ATSP may be able to transmit the 
scheduled time of arrival to an aircraft and allow 
the flight crew to incorporate this constraint as a 
“required time of arrival” (RTA) in their 
trajectory planning. This type of constraint puts 
time pressure as well as positional endpoint 
requirements on the current segment of flight. 

Airspace hazard constraints are present when 
certain regions of airspace are inadvisable for 
entry. Examples of such airspace hazards are 
active special-use airspace (SUA) and 
convective weather cells. The fomrer constraint 
is proscribed by the ATSP or aeronautical 
databases, and the user (company or pilot) 
generally proscribes the latter constraint based 
on safety. Regardless, the pilot must take these 
airspace restrictions into account when defining 
conflict resolution strategies. 

Performance constraints include restrictions 
based primarily on the operating limitations of 
the aircraft. Restrictions such as maximum 
operating altitude, speed, or climb/descent rate 
govern the degrees of freedom available for 
conflict resolution maneuvers. 

Economic constraints include user-generated 
operational guidelines that must generally be 


met a majority of the time for a commercial 
aviation business to remain viable for the long 
term. Examples include fuel efficiency, 
schedule considerations, and passenger comfort. 

When constraints of these types are 
considered in combination with the task of 
separation assurance, the type of traffic 
surveillance information provided to the flight 
crews may play a more critical role in their 
ability to repeatedly and reliably meet their 
separation assurance responsibility than in 
unconstrained operations. This issue extends 
beyond the minimum requirement for 
information that enables airborne separation 
assurance, and it extends beyond the preferences 
of the flight crew. The study of constrained 
operations is critical to determining the overall 
advisability of exchanging intent information to 
enable the participants of future distributed- 
responsibility operations to achieve all 
objectives and meet all constraints with long- 
term stability. 

This paper presents the design and some 
preliminary findings of an experimental study 
designed to provide insight into the issue of 
intent information exchange in constrained en- 
route operations. In the experiment, two modes 
of autonomous aircraft operation, tactical and 
strategic, were tested for comparison. Both 
modes have been proposed as viable alternatives 
in the constrained, en-route, operational 
environment described earlier. The tactical 
mode is primarily characterized by minimal 
information exchange, i.e., state information 
only. The strategic mode features the exchange 
and use of both state and intent information. 
Additional important distinctions between these 
modes exist and are described in detail in 
“Modes of Autonomous Operation.” Sixteen 
active line pilots each flew a simulated aircraft 
through four en-route scenarios: the two modes 
of operation within two levels of operational 
complexity. In these scenarios, the pilots 
encountered traffic conflicts and operational 
constraints of the types described earlier. 
Automated conflict management tools (for 
conflict prevention, conflict detection, and 
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conflict resolution) were provided as appropriate 
to each operational mode. The pilots were 
instructed to resolve traffic conflicts and 
simultaneously meet the operational constraints. 
Data acquired for analysis included objective 
measures related to trajectories flown, the pilot’s 
manipulation of the display features, the timing 
of conflicts and maneuver decisions, and pilot 
workload. Subjective measures of workload and 
pilot assessments of display features and concept 
feasibility were also acquired. The experiment 
was conducted in the NASA Langley Air Traffic 
Operations Laboratory, a medium-fidelity 
workstation simulation of airspace operations. 

Modes of Autonomous Operations 

Two modes of autonomous operations have 
been proposed as viable alternatives for free 
flight. They differ from each other in several 
respects, beyond just the level of information 
exchange. 

Tactical mode 

The tactical mode has been developed and 
investigated over several years in batch and 
piloted simulation studies by the NLR (National 
Aerospace Laboratory of The Netherlands) 3 , and 
it is primarily characterized by simplicity in 
several respects. Its design attempts to minimize 
the requirements placed on supporting 
technology, including both data link and pilot 
decision-support automation. Broadcast data- 
link bandwidth requirements are minimized by 
employing conflict detection based only on the 
current aircraft state vector (current position, 
altitude, ground track, ground speed, and vertical 
speed). On-board conflict detection algorithms 
deterministically compare (in the current 
implementation) the state vector of traffic 
aircraft with that of the own-ship. To minimize 
false alerts associated with extrapolation errors 5 , 
state-vector-based conflict detection is limited in 
its “look ahead” time horizon. The research of 
NLR determined that 5 minutes was sufficient 
for separation assurance, and so a 5 minute look- 
ahead horizon was used in the current study for 
state-based conflict detection. 


If a conflict (i.e., predicted loss of regulatory 
separation) is detected, the pilot is alerted to the 
event and the conflict resolution algorithm is 
automatically activated to calculate maneuver 
advisories for the pilot. These conflict 
resolution advisories are simple in that they are 
recommended changes to the own-ship state 
vector (i.e., change in track, ground speed, or 
vertical speed). The pilot implements the 
maneuver by setting heading, airspeed, or 
vertical speed targets in the Flight Control Panel 
(FCP) to match the advised settings. This 
procedure is comparable to the pilot's current 
use of the FCP to comply with a vector for 
traffic issued by Air Traffic Control (ATC). 
Concurrently, a conflict prevention system (i.e., 
PredASAS, described earlier) monitors all 
possible single-dimensional maneuvers for 
conflicts, and it indicates to the pilot what 
maneuvers shall not be made to prevent new 
conflicts from occurring, essentially a “no-go” 
alerting system. All maneuvers outside of the 
displayed no-go bands are conflict-free for at 
least the next 5 minutes, assuming the traffic 
aircraft do not maneuver during this time. 

The resolution maneuvers in this 
operational mode are tactical in nature because 
they only resolve the conflict and do not account 
for a rettim to the original flight plan or the 
consideration of external constraints such as 
RTAs or airspace hazards. This liighlights the 
primary characteristic of the tactical mode, that 
of the open-loop (manual) nature of meeting 
constraints. It is hypothesized that the pilot 
would typically solve problems sequentially: 
first - resolve the conflict by maneuvering clear; 
second - avoid any nearby airspace hazards; 
third — develop an efficient plan to return to 
course; fourth - make adjustments to meet RTA 
and other ATC constraints. This approach has 
the effect of spreading decision-making over 
time and possibly simplifying the maneuver 
decisions. After maneuvering safely to resolve 
the original conflict, the pilot monitors the 
PredASAS information to determ in e when it is 
safe to return to course. 
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Strategic Mode 

The strategic mode is a closed-loop 
(automated) method of trajectory planning. Any 
trajectory changes implemented by the flight 
crew will have been determined a priori to meet 
all known constraints and optimization criteria 
while both solving the current problem (e.g., a 
detected conflict) and returning the aircraft to 
course. This approach places greater demand on 
decision- support automation in that it must 
generate trajectories for pilot review based on a 
simultaneous solution of constraints and 
objectives. To ensure that the entire modified 
portion of the flight plan will be conflict free, 
the automation must have a longer look-ahead 
time horizon to be aware of future changes in 
traffic-aircraft state vectors. The intended 
trajectory or “intent” of each aircraft is therefore 
included in its broadcasted data-link message, 
and greater demands are correspondingly placed 
on the data link bandwidth to accommodate the 
additional information. For this experiment, the 
intent message was defined to be a series of 
trajectory change points, although other forms of 
intent are acknowledged to also be pertinent and 
are currently being studied 6 . 

Conflict detection is typically performed 
using both traffic state information and traffic 
intent information. The state-based conflict 
detection in the experiment was identical to that 
in the tactical mode with a 5 -minute look-ahead 
horizon. The intent-based conflict detection 
deterministically compared the own- ship flight 
plan to the traffic-aircraft broadcast intent in a 
search for intent conflicts 7 . Whereas a 15-20 
minute look-ahead horizon is thought to be 
appropriate for intent-based conflict detection, a 
shorter horizon of 8 minutes was used in the 
current study allowing more data mns to be 
accomplished during the limited availability of 
the subject pilots. 

A conflict-alerting decision algorithm was 
developed to determine when and how to alert 
the pilot to potential and actual conflict 
situations where state and intent data are both 
used for detection. The utility of combining 


state-based and intent-based conflict detection is 
that it allows the alerting system to distinguish 
between a full-fledged conflict alert that requires 
own-ship action, and a situation that should be 
resolved by the traffic aircraft. The latter 
situation would require no own-ship action but 
would have the potential for elevating to the 
former category. Events where this could occur 
include failure of either aircraft to observe 
priority and/or maneuver flight rules (described 
below) or an unannounced deviation from the 
broadcast intentions (i.e., intent non- 
conformance). It is hypothesized that 
distinguishing within the alerting logic between 
situations that require or do not require own-ship 
action should reduce unnecessary maneuvering 
and therefore improve overall system stability. 
The alerting logic is described in more detail 
later in “Flight Deck Display Design.” 

In contrast to the tactical mode, the strategic 
mode takes advantage of existing technology on 
the flight decks of many commercial aircraft for 
conflict resolution, in particular the FMS 
because it contains a detailed database of the 
aircraft performance characteristics and 
operating limitations. By coordinating the 
conflict-resolution calculations with the flight 
planning and trajectory generation functions of 
the FMS, a complete re-planning of the local 
trajectory can be performed that guarantees that 
the new trajectory is within the flight envelope. 
In addition, the FMS can also be used to close 
the loop on ATSP constraints. Speed and path 
strategies that meet an RTA at a downstream fix 
or airspace boundary can be incorporated into 
the proposed conflict-resolution trajectory. 
Resolution strategies can also incorporate 
predicted locations of convective weather cells 
and scheduled activation of special-use airspace 
or any region that would be considered 
hazardous or inadvisable to enter, assuming this 
information was made available to the aircraft 
systems (which may require additional flight- 
deck and ground-based functions). Since the 
solution space that meets these constraints 
would normally be large, trajectory optimization 
can be perfomied to achieve a desired goal, such 
as fuel economy, a comfortable ride, or an early 


6 



arrival. The FMS can then be used to fly the 
complete resolution trajectory, potentially 
reducing the workload of the flight crew. 

Conflict resolution advisories for conflicts 
based on valid intent (i.e., the intruder aircraft is 
determined to be conforming to its broadcast 
intent) were calculated using a genetic- 
algorithm-based optimization routine 8 . This 
routine was designed to iterate trajectory 
constraints with the FMS until a conflict-free 
trajectory that meets all additional constraints 
(e.g., RTA) is determined. Further iterations are 
then performed to optimize a selected parameter 
(e.g., fuel-bum minimization). The trajectory 
would normally be flown by the FMS. For 
conflicts requiring own-ship action that are 
based on state information, resolution advisories 
identical to those in the tactical mode are 
presented. These advisories are flown by the 
pilot using the FCP. 

Flight Rules for the Strategic Mode 

Two types of flight rules are envisioned for 
the strategic mode of operation, each providing a 
distinct benefit. A maneuver flight rule is one 
that governs what types of maneuvers are not 
permissible in certain situations. The strategic 
mode incorporates a maneuver flight rule that is 
designed to prevent near-temi conflicts from 
suddenly appearing. The same rule was applied 
to the tactical mode, as described earlier in the 
use of the PredASAS alerting system. The rule 
states that an aircraft may not implement a 
change in track, speed, or vertical speed that 
creates a near-temi conflict (for the current 
study, within 5 minutes). The pilot would meet 
the requirements of this rule by avoiding flight 
in the direction of a PredASAS band, although 
transition through a band is pemiitted. This 
maneuver flight rule has the additional benefit of 
providing some predictability of autonomous- 
aircraft operations, which should aid the ATSP 
in developing stable strategies for managed- 
aircraft separation. 

A priority flight rule defines which aircraft in 
a given conflict situation is responsible for 


resolving the conflict. The tactical operational 
mode has no priority flight rule in that it 
assumes every autonomous aircraft shares equal 
responsibility to resolve conflicts, which is 
prudent given the short time horizon for 
detecting and resolving conflicts. The strategic 
operational mode also assigns equal 
responsibility for near-term conflicts (defined 
for the current study as 5 minutes). However for 
conflicts more than 5 minutes away, the conflict 
geometry is used to determine a “right-of-way” 
priority, such that one aircraft must give way to 
the other. By assigning resolution responsibility 
to one aircraft in a conflict pair, predictability 
should increase, total maneuvering at the system 
level should decrease (ideally by one-half since 
only one aircraft maneuvers), and system-level 
traffic flow stability may be enhanced. For 
conflicts detected significantly far in advance 
(perhaps greater than 15 minutes - a subject of 
future research), the benefits of assigning 
responsibility are likely to disappear, and 
therefore priority flight mles would no longer be 
applied. The application of flight mles as a 
function of time is shown in figure 1 . 

Flight Deck Display Design 

A new cockpit display of traffic information 
(CDTI) design concept, exercised in the strategic 
mode for this experiment, was developed to 
address the issues of effectively integrating 
(rather than superimposing) state and intent 
information for conflict detection into a single 
presentation. The design, summarized below, 
was built on state-only and intent-only display 
features previously developed by NLR’ and 
NASA Ames Research Center. The aircraft 
simulation used in the current experiment was a 
representation of the MD-11 aircraft. The new 
display features for autonomous operations were 
therefore integrated into the MD- 1 1 flight-deck 
display suite, and existing MD-11 conventions 
were adhered to as much as possible. The 
Primary Flight Display (PFD) and Navigation 
Display (ND) were the only displays affected. 
An ND control panel was added, and its use is 
described below. 
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CDTI Design Features 

The new display design follows the common 
approach of superimposing traffic data on the 
ND. The ND with some of the CDTI features is 
shown in figure 2. An unfilled chevron symbol 
represents the position and track angle of a 
traffic aircraft relative to the own-ship. A short, 
protruding stinger indicates the aircraft is 
operating in autonomous status, and the absence 
of the stinger represents managed status. The 
symbols are color encoded for at-a-glance 
relative altitude information: blue for aircraft 
above the own-ship; green for aircraft below; 
and white for same altimde within ±1000 ft. 
Attached to the base of each chevron is the 
altitude value in 100s of feet; the pilot is able to 
select whether absolute or relative altitude is 
shown using a button on the ND control panel. 
An up or down arrow is also shown if the 
climb/descent rate exceeds 100 feet per minute. 
Through the ND control panel, the pilot is able 
to filter the display of aircraft (for de-cluttering) 
that are outside a pilot-selectable vertical range. 
Pilots could input values from ± 2000 feet to ± 
40,000 feet, or they could turn the filter off and 
display all aircraft within the horizontal range 
selected. Horizontal range display information 
could be varied from 320 nautical miles to 10 
nautical miles. To simulate airborne broadcast 
datalink range limitations, only information 
from aircraft within 120 nautical miles were 
received and displayed. 

Additional infomration is available to the 
pilot for either all aircraft on the display or just 
individual aircraft. For all aircraft, the pilot can 
select to display the call sign next to each 
chevron. The pilot is also able to display the 
state vector extending in front of the chevrons 
for pilot-selectable lengths representing from 2 
to 20 minutes of flying time. For any single 
aircraft, the pilot is able to select additional 
information to be displayed, including: a data 
block showing call sign, absolute altitude, target 
altitude (if available), and ground speed; the 
path representing the broadcast intent (if 
available); and the protected zone to be avoided 
for separation assurance. 


The display design incorporates the 
PredASAS conflict prevention system. This 
system consists of “no-go” indications in the 
form of color-coded bands on the ND compass 
rose, the PFD vertical speed scale, and PFD 
airspeed tape. The bands indicate which 
instantaneous changes to the aircraft state vector 
would result in a state conflict. An amber color 
indicates a loss of separation would occur in less 
than 5 minutes, and a red color indicates a loss 
in less than 3 minutes. To identify which 
aircraft is causing the band, the pilot uses a 
point-and- select device (the computer mouse for 
the current desktop simulation experiment) to 
select the band on the ND compass rose. A 
circle representing the protected zone of the 
aircraft is displayed for 3 seconds. For aircraft 
outside the vertical filter setting, the aircraft and 
its protected zone are displayed for 3 seconds 
despite the filter. 

Conflict Alerting 

For conflict situations, the alerting of the 
flight crew uses an approach that is similar to the 
MD-11 convention for aircraft system alerts. 
The alerting logic is based on three levels of 
alerting, and the symbology is shown in figure 3. 

A Level 1 alert is used when infomration 
must be conveyed to the pilot, but no action is 
required. The symbology for a Level 1 alert is a 
change in the traffic-aircraft chevron color to 
amber, with the chevron remaining unfilled. 
This alert level is used primarily in two 
situations. In the first situation, a state conflict 
is detected, but no intent conflict is detected 
(i.e., the aircraft is planning to change course or 
altitude before losing separation). If both 
aircraft are determined to be conforming with 
their broadcast intent, then no action is required 
other than to continue to monitor for intent 
conformance. In the second siuiation, an intent 
conflict is detected, but the priority flight rules 
decree that the own-ship has priority and the 
traffic aircraft must maneuver. Again, no action 
is currently required, and the traffic aircraft is 
“pointed out” to the own-ship pilot. The Level 1 
alert may be useful in "priming” the pilot for 
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potential blunder situations that would elevate 
the alerting to Level 2. Note that Level 1 alerts 
only occur in the strategic mode. 

A Level 2 alert requires action by the own- 
ship flight crew. The alert is used when a 
conflict has been detected, and it is the 
responsibility of the own-ship flight crew to 
resolve the situation. The symbology for a 
Level 2 alert has two components separated in 
time. When the conflict is first detected, an 
aural alert is given, and the traffic aircraft 
symbol changes to a filled, amber chevron. The 
predicted position of the traffic aircraft at the 
time of loss of separation is shown with a circle 
representing its protected zone. The flight plan 
or state vector is shown in amber between these 
two symbols. If the time to loss of separation is 
less than 3 minutes, additional alerting text and a 
countdown time is shown on the ND, as well as 
an additional aural alert. 

A Level 3 alert requires immediate action by 
the own-ship flight crew. This alert corresponds 
to the actual loss of separation. The symbology 
for a Level 3 alert is a red filled chevron for the 
traffic aircraft, plus an aural alert. 

Conflict Resolution Advisories 

The detection of a conflict triggers the 
calculation of a resolution advisory by the 
decision-support automation, and the proposed 
trajectory is sent directly to the FMS. The 
proposed trajectory is displayed on the ND for 
pilot review. As a temporary measure, 
conventional symbology for an alternate route is 
used for the proposed trajectory. The calculation 
and display of the proposed trajectory occur 
automatically. A Control and Display Unit 
(CDU) page was devised for executing (or 
rejecting) the trajectory. 

If the conflict persists until it is detected by 
the comparison of the state vectors, which 
indicates that loss of separation is predicted to 
occur within 5 minutes, a set of tactical- 
maneuver options are displayed to the pilot as a 
safety enhancement that pemiits immediate 


conflict resolution with simple maneuvers. 
These tactical advisories are shown concurrently 
with the alternate FMS-route advisory, 
providing the pilot a quick, tactical option to 
clear the conflict alert while providing more 
time to review the proposed changes to the FMS 
route. An amber heading bug on the ND 
compass rose and a corresponding airspeed bug 
on the PFD airspeed tape indicate a combined 
heading/airspeed combination that resolves the 
conflict. Additionally, an amber bug on the PFD 
vertical speed scale indicates to the pilot an 
alternative vertical maneuver that also would 
resolve the conflict. 

These CDTI design features were intended to 
enhance the pilot performance in autonomous 
operations. The conflict detection and resolution 
design features were intended to maximize the 
pilot’s situational awareness of air traffic and 
airspace hazards while minimizing the 
monitoring and cognitive workload. The 
enhanced ND tools allowed the pilot to 
determine each aircraft’s call sign (for possible 
aural or digital data link), predict its future state, 
avoid conflicts (using the prevention bands), and 
maintain self-separation using the alerting 
functions. These tools were evaluated to 
determine their usability under the multiple 
constraints of airborne self-separation, meeting 
RTAs, flying efficiently, and maximizing 
passenger comfort. 

Air Traffic Operations Laboratory 

The experiment was conducted in the Air 
Traffic Operations Laboratory (ATOL) at the 
NASA Langley Research Center. The ATOL 
hosts a workstation-based human-in-the-loop 
simulation of air traffic operations. The 
simulation consists of the Traffic and Events 
Manager (TMX), developed by NLR and NASA 
specifically for free-flight research, and the 
Aircraft Simulation for Traffic Operations 
Research (ASTOR), a workstation flight 
simulator for transport-category FMS-equipped 
aircraft under development at the NASA 
Langley Research Center. In this environment, 
multiple human pilot research subjects can “fly” 
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several ASTOR aircraft, interacting in traffic 
scenarios hosted by TMX. Incorporated into 
ASTOR are the following automation functions: 
conflict prevention advisories; conflict detection, 
alerting, and resolution; airspace hazard 
detection, alerting, and resolution; trajectory 
generation that meet RTA and performance 
constraints; and auto-flight systems with FCP or 
FMS guidance. ASTOR and TMX continuously 
log detailed trajectory and event data for 
analysis. 

Experiment Objectives and 
Approach 

The primary objective of the current 
experiment was to compare the two proposed 
operational modes applicable to airborne 
separation assurance in a constrained en-route 
environment. The experiment focused on 
operational aspects that relate to commercial- 
transport autonomous (free maneuvering) 
aircraft as defined by the DAG TM Concept 
Element 5 10 . A second objective was to assess 
the usability of the flight-deck display and user- 
interface design that integrated state-based and 
intent-based traffic infomiation to reinforce pilot 
situation awareness. 

This current research activity was focused on 
the operations of a single autonomous aircraft in 
en-route cmise flight with variable airspace 
complexity (i.e., traffic density, weather cells, 
SUA). Beyond the current research scope were 
direct interactions with the ATSP, managed 
aircraft, or other piloted autonomous aircraft. 
The study did not address multi-person flight 
crews, crew resource management, or voice 
communications. Climbs and descents of the 
own-ship were not studied, nor were the effects 
of winds or failure modes of decision-support 
automation or CNS infrastmcture. 

The comparison of the two proposed 
operational modes addressed the hypothesis that 
the strategic mode has the following 
characteristics relative to the tactical mode: 

• Reduced the sensitivity of safety and 


efficiency metrics to operational complexity 

• Impacted pilot cognitive workload and time- 
on-task in maintaining separation 

• Increased pilot confidence in the 
information and advisories provided by the 
decision-support automation 

• Increased pilot acceptance of the expansion 
of the pilot’s role to include separation 
responsibility 

A 2-by-2 within-subjects experimental 
design was used to address the research 
objectives. The primary independent variables 
were operational mode (tactical and strategic) 
and operational complexity (low and high). 

Operational complexity, for the purposes of 
this experiment, was assumed to be a function of 
traffic density and airspace hazard density. 
Research has shown that traffic density is 
correlated with operational complexity". 
Airspace hazard density was added as an 
additional relevant complexity factor of 
constrained operations. Although a meaningful 
quantification of the aggregate complexity is 
difficult to establish, these two complexity 
factors were simultaneously set at relatively low 
and high conditions to represent a combined 
operational complexity that was either relatively 
benign or fairly challenging. The high 
complexity condition, however, was not so high 
that reasonable trajectory solutions that met all 
constraints were unavailable. 

Scenarios of traffic aircraft were created to 
represent U.S. traffic distribution and patterns 
that might be expected in a DAG-TM 
environment. An experiment region was defined 
by parallels 27°N and 39°N and meridians 89°W 
and 105°W, roughly a 600nm by 700nm area 
centered on Fort Worth Center. Recorded U.S. 
flight data from peak traffic hours on November 
12, 1997 were analyzed for departure-arrival city 
pairs and average rates. The recorded 
trajectories were replaced with roughly direct 
routing between the city pairs, and routes 
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outside the experiment region were deleted. To 
create the background traffic (i.e., aircraft not 
involved in conflicts with the own- ship) for the 
low complexity scenarios, aircraft were 
launched between the same city pairs using the 
new routing at average rates detemiined from 
the recorded-data analysis. For the high 
complexity scenarios, the rates were tripled. 
Random high-altimde cruise levels were 
assigned. Polygons representing special use 
airspace and hazardous weather were added to 
the background traffic patterns. The polygons 
were generally small and widely scattered. 

The background traffic included both 
autonomous and managed aircraft, and the pilots 
were able to distinguish between them using the 
traffic display, even though the conflict 
situations in this experiment did not directly 
involve managed aircraft. The purpose for 
including managed aircraft was to maintain 
reasonable consistency in the operational 
environment represented across multiple 
experiments and thereby facilitate inter- 
experiment data analysis. Future experiments 
will address issues directly related to 
interactions between autonomous and managed 
aircraft. An 80/20 ratio of autonomous-to- 
managed aircraft status was used, representing a 
reasonably matured end state for the DAG TM 
CE-5 concept, which does not assume 100 
percent autonomous operations will ever be 
reached. None of the background aircraft 
maneuvered to resolve conflicts amongst 
themselves 

Sixteen active commercial transport pilots 
participated in the study. Each flew a scenario 
in each of the four conditions represented in the 
2-by-2 experimental design described above. 
The scenarios depicted the en-route phase of 
flight, and each scenario consisted of three 
segments (i.e., flight legs), and each segment 
contained a conflict situation. Three types of 
conflict situations were used in this experiment, 
one per segment. The conflict types are 
genetically depicted in figure 4. The subject 
aircraft ("own-ship") is on the left, and the 
traffic aircraft ("intruder") is on the right. Solid 


lines indicate intent, and the dashed line 
indicates a deviation from intent (i.e., a blunder). 
The “state-only” conflict occurs when only the 
state trajectories threaten a conflict. The “intent- 
only” conflict occurs when only the intent 
trajectories threaten a conflict. The “blunder” 
conflict is similar to the “state-only” conflict, 
but the intruder aircraft does not adhere to the 
planned trajectory change in the broadcast intent 
message, and the aircraft remains in conflict. 
These conflict types are a subset of those that 
could occur between aircraft. The chosen types 
were anticipated to have the most noticeable 
effect on pilot decision-making. Also, for each 
conflict design, the own-ship’s intent and state 
trajectories coincided, and therefore only 
trajectory changes for the traffic aircraft were 
addressed in order to reduce the data set to a 
manageable size. These conflict situations were 
distributed in both the horizontal and vertical 
planes, and the initial relative orientation of the 
intruder aircraft with respect to the own-ship 
was varied. 

Each of the three segments within a scenario 
was terminated with an RTA constraint. The 
subject pilot was tasked to ensure separation 
from the traffic aircraft while meeting the RTA 
constraint. The subject pilot was also told to 
avoid airspace hazards. In order to assess 
workload impact, the subject pilot was given a 
secondary task involving periodic monitoring 
and reporting of aircraft system status. 
Additionally, the pilot was prompted every two 
minutes to record his real-time assessment of 
workload on a seven-point scale from very low 
to very high. 

Recorded objective measures included flight 
plan infomiation, predicted and actual 
trajectories, event times for conflict alerting, 
pilot response times to alerts, conflict resolution 
maneuver type/initiation, RTA variances, and 
secondary task performance latency/accuracy (as 
an objective workload measure). In addition, 
frequency and ease of infomiation use were 
collected (via electronic recordings, 
observations, and questionnaires) to evaluate the 
usability of the user interface. 
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Subjective measures were obtained through 
post-scenario interviews and questionnaires. 
This information was used to gain insight into 
pilot confidence in automation, perceived 
workload, and role acceptance in autonomous 
operations. 

Preliminary Results and Discussion 

Preliminary results are presented in two 
parts. In the first part, three case studies from 
the recorded data set are presented to illustrate 
aspects of the tactical and strategic modes as 
flown by subject pilots in the experiment. In the 
second part, subjective ratings and comments by 
the pilots comparing the tactical and strategic 
modes are given. These data represent only a 
fraction of the total data acquired, and further 
analysis and reporting of additional results is 
planned. 

Case Studies 

Some of the differences seen between the 
two modes of operation, tactical and strategic, 
will be illustrated by showing how various pilots 
solved the same traffic situation using the tools 
and procedures associated with each mode. 
These illustrative flights have been chosen to 
highlight some of the differences between the 
two modes and do not necessarily represent 
typical performance by all of the pilots. In 
general, it was found that when a pilot fully 
utilized the set of tools offered, they were able to 
successfully complete their tasks of maintaining 
separation, meeting an RTA, and operating in an 
efficient manner. There were only two out of 
192 experiment segments where the pilot lost 
separation; one in each mode of operation. 
Interestingly, both occurred in low complexity 
airspace. In one case, the pilot made an altitude 
change without considering the conflict 
prevention bands on his vertical speed indicator 
and descended into a very near-temr conflict. 
Minimum separation in this case was 
approximately 4.9 nrn and 900 ft, just within the 
proscribed minima of 5 run and 1000 ft. In the 
other case, the pilot tried to avoid a conflict by 
maneuvering between two approaching aircraft. 


When one of these aircraft Uimed towards him, 
the pilot was unable to react quickly enough to 
maintain separation. Closest approach was 3.8 
nm. Both simations may have developed 
through a lack of familiarity with the tools and 
experience in self-separation simations. 

Case 1: State -only conflict 

The first case study will illustrate how two 
pilots reacted to a state-only conflict. In this 
conflict type, the state vectors of the intruder and 
the own-ship are initially in conflict, i.e., 
threaten a loss of separation. However, the 
intruding aircraft has a trajectory change point 
(TCP) in its flight plan that occurs between three 
and five minutes before loss of separation which 
would take the aircraft out of conflict. 
Therefore, in a state-only conflict, if both 
aircraft follow their flight plans, there will be no 
intrusion. Both of the subject pilots in this case 
study saw exactly the same conflict geometry 
with identical background traffic and airspace 
hazards. The only difference was the mode of 
operation. 

Pilot 1 operated in the strategic mode. The 
relevant recorded tracks are shown in Figure 5. 
Other background aircraft and airspace hazards 
are not shown. During the first several minutes 
of the scenario, Pilot 1 made use of his ability to 
view the flight plan of AA552. He could see the 
upcoming TCP and detemiined that AA552 
would pass behind his aircraft and was therefore 
not a threat. At 6:35, Pilot 1 received a Level 1 
traffic advisory on C0755, the intended intruder 
(Figure 5, position A). By checking the flight 
plan he could see that C0755 has a turn planned. 
The Level 1 advisory indicated that if both 
aircraft continue along their flight paths, there 
would be no loss of separation and therefore no 
action was required. Over the next two minutes, 
the pilot monitored the progress of C0755 with 
the flight plan displayed. Once the intruder 
started its turn (and the state vectors were no 
longer in conflict), the Level 1 advisory 
disappeared (position B). For the remaining 
seven minutes of this segment. Pilot 1 continued 
to scan the traffic for possible conflicts. The 
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pilot passed over the RTA waypoint at the 
proper altitude and on time. No maneuver by 
the pilot was required during the segment. He 
was able to complete all of his secondary tasks 
in a timely manner. His self-assessment of 
workload level was consistently low. 

In comparison to events of Pilot 1 operating 
in the strategic mode, the tactical mode is 
illustrated in how another pilot (Pilot 2) flew the 
identical segment but with use of the tactical- 
mode tools and procedures. Therefore, he did 
not have the flight plan available for other 
aircraft; nor did he have access to a closed-loop 
resolution if a conflict were detected. The 
relevant recorded tracks are shown in Figure 6. 
A little over a minute into the scenario, Pilot 2 
climbed 1000 ft from his starting altitude of 
Flight Level (FL) 320 (Figure 6, position A). 
This was likely in response to FX281 who was 
crossing 18 nm in front of the own-ship (aircraft 
flown by the subject pilot). At four minutes into 
the scenario (4:00), Pilot 2 was observed to be 
studying AA552 who was flying opposite 
direction, 14 nm to the right. At 5:20, AA552 
made a 45 degree turn towards the own- ship 
(position B). At the turn, AA552 was nine miles 
in front and fourteen miles to the right of the 
own-ship, and its trajectory would take it well 
behind the own-ship. However, the unexpected 
course change of AA552 seemed to have 
unsettled Pilot 2, as he reacted to a situation that 
was not a threat by immediately turning 8 
degrees left, away from AA552 (position C). At 
6:00, Pilot 2 descended back to FL320 (position 
D), and at 7:00, the pilot re-engaged the FMS 
lateral navigation (position E). As the aircraft 
turned right to recapture the original flight plan, 
its state vector came into conflict with C0755, 
the intended intruder, crossing from the right at 
FL320. Within a few seconds, Pilot 2 turned 20 
degrees more to the right to pass behind C0755 
(position F). However, the intruder had an 
upcoming TCP (unknown to Pilot 2) that would 
result in a left turn towards the own- ship. As 
C0755 started its turn, Pilot 2 decided to ascend 
back to FL330, 1000 ft above C0755 (position 
G). Three minutes later, as Pilot 2 was trying to 
recapture his flight plan, there was a conflict 


alert with FL (AirTran) 688. As the pilot 
maneuvered to the right to resolve this conflict 
(position H), another conflict arose on his left 
(TW587). One maneuver resolved both 
conflicts. One minute later, the pilot re-engaged 
FMS lateral navigation. This turn toward the 
flight plan put the own-ship back into conflict 
with both FL 688 and TW587. Pilot 2 turned 
back to the right to avoid these aircraft (position 
I). With only forty seconds before the RTA, 
Pilot 2 turned back towards his flight plan and 
started to descend to 32000 ft (position J). Due 
to excessive maneuvering, the pilot was unable 
to meet the time and altitude constraint at the 
final waypoint of the scenario. This pilot 
encountered a total of six separate alerts on three 
different aircraft where only one was intended. 
Pilot 2 also failed to complete three out of five 
secondary tasks (aircraft systems monitoring and 
reporting). His self-assessment of workload was 
consistently higher than Pilot 1 throughout the 
segment. 

This case study was an illustration of a 
conflict type for which the preferred course of 
action would be to take no action other than 
monitor the other aircraft. Intent information 
was required to determine that separation would 
be maintained if the aircraft conformed to their 
intent. State information was required to 
provide the traffic advisory information and to 
monitor for conformance. The scenario 
illustrates a benefit of combining state and intent 
information in conflict alerting. Without this 
approach, as demonstrated in the case study, a 
pilot may be subjected to unnecessary problem 
solving. 

Case 2: Intent-only conflict 

In the intent-only conflict type, the aircraft 
state vectors are not initially in conflict. 
However, a planned TCP by the intmder aircraft 
results in trajectories that threaten a loss of 
separation. Conflict detection systems that use 
the intent information are able to detect the 
conflict before the TCP maneuver, whereas 
state-based systems would not alert the pilot 
until the TCP maneuver has been completed. 
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In the following case study, AA686 was the 
intended intruder and was climbing towards its 
planned cruise altitude that coincided with that 
of the own-ship. The level off would occur 
approximately four minutes before loss of 
separation. 

Pilot 3 flew the scenario in the strategic 
mode. The relevant recorded tracks are shown 
in Figure 7. The pilot was initially observed to 
be studying the intent (i.e., the broadcast portion 
of the flight plans) of many of the aircraft. At 
5:45 (Figure 7, position A), a Level 2 traffic 
alert indicated a conflict with AA686, with loss 
of separation to occur over seven minutes into 
the future. A resolution trajectory calculated by 
the automation was displayed to the pilot, 
indicating an additional waypoint that would 
avoid loss of separation with AA686 and return 
the own-ship to the RTA waypoint. Pilot 3 
accepted the resolution advisory through the 
CDU, and the flight plan on the ND adjusted to 
show the modified route (position B). The alert 
symbology disappeared when the resolution 
advisory was accepted. Over the next several 
minutes, the pilot was observed to watch the 
passage of AA686 at very low ND range settings 
to verify that the new trajectory avoided loss of 
separation with AA686. This indicated that 
while this particular pilot was willing to accept 
the offered resolution, he did not fully trust the 
system and felt compelled to closely track the 
separation until AA686 had passed behind him. 
Pilot 3 was able to meet all given constraints. 

Pilot 4 flew the same scenario as Pilot 3 but 
was given the tactical tools and procedures to 
use. The primary difference in the available 
tools was that Pilot 4 could only opt to see the 
state-vector extrapolation of each aircraft, not its 
current intent that, in the case of the intmder, 
was a target altitude of FL320. The relevant 
recorded tracks are shown in Figure 8. The first 
several minutes of the segment progressed 
similar to that for Pilot 3. Pilot 4 received his 
first alert on AA686 at 9:20 when AA686 
leveled off at FL320 (Figure 8, position A). 
This was three and a half minutes later than Pilot 
3 was alerted to the conflict. Pilot 4 elected to 


immediately turn to the left (position B). 
Several seconds later, Pilot 4 climbed to FL330 
(position C), possibly because he was unsure 
whether the turn would be sufficient to resolve 
the near-term conflict. While climbing (11:30 
simulation time), an alert appeared on another 
aircraft flying opposite direction at FL365. 
Between this time and 12:40 the pilot made 3 
major heading changes: right, left, and then right 
(position D). It is unclear why the pilot made 
three heading changes instead of just one. As 
the original intmder, AA686, passed beneath 
own-ship, Pilot 4 maneuvered to recapture his 
lateral path. At 14:30, Pilot 4 descended back to 
FL320 (position E). He was able to successfully 
maintain separation and reach the altitude and 
time constraints at the RTA waypoint. 

In this case study, Pilot 3 benefited from the 
strategic mode in three ways: knowing the 
intentions of the intmder aircraft, having plenty 
of time to determine a satisfactory resolution, 
and having an automated system load a viable 
solution into the FMS for review and 
acceptance. Pilot 4 had little notice of the 
conflict, and thus may have felt pressured into 
making quick maneuver decisions before 
determining what other conflicts may result 
from the maneuvers. Research at the NLR has 
suggested that intent conflicts such as this would 
not normally be an issue, provided that each 
aircraft is equipped with a conflict prevention 
system such as PredASAS and that conflict- 
generating maneuvers (such as the level-off of 
AA686 in this scenario) are not permitted 3 . For 
this to be a viable approach, the conflict 
prevention system may need to be integrated 
with the FMS in order to override such 
maneuvers. 

Case 3: Blunder conflict 

The third conflict type that was presented to 
the pilots was a blunder, or non-conformance, 
conflict. The scenario geometry is similar to the 
state-only conflict, but in the blunder case, the 
approaching aircraft fails to maneuver at the 
TCP. A loss of separation would therefore occur 
if the own-ship pilot fails to maneuver, hi the 
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experiment, the TCP was placed three and a half 
minutes prior to loss of separation. 

The first example for the blunder conflict 
will be the strategic mode. The relevant 
recorded tracks are shown in Figure 9. For the 
first several minutes of the flight, Pilot 5 was 
observed to carefully scmtinize the traffic data 
with frequent changes to the ND range. At 6:30, 
a Level 1 traffic advisory alert on AC303 was 
displayed, indicating a possible threat but no 
action currently required (Figure 9, position A). 
The pilot immediately displayed the flight plan 
for this aircraft. At 8:45 AC303 blundered 
through its TCP, failing to follow its broadcast 
intent; the alert changed to a Level 2 alert which 
indicates that action will be required (position 
B). Two seconds later, the pilot initiated a 16 
degree heading change away from the intruding 
aircraft (position C). Over the next three 
minutes, Pilot 5 made four minor heading 
changes to minimize the distance between 
himself and the intruder, essentially fine-tuning 
the resolution for minimum path deviation. The 
closest approach point was 5.1 nm (position D). 
After the intruder had passed behind own-ship, 
the pilot engaged FMS navigation to recapture 
his flight plan (position E). He successfully met 
the time and altitude constraints at the RTA 
waypoint. Although Pilot 5 was able to view the 
flight paths of the traffic aircraft, he only 
requested this information once, when he 
received the Level 1 alert on AC303. The pilot 
was consistently late performing the secondary 
task. 

Pilot 6 flew the same blunder scenario in the 
tactical mode. The relevant recorded tracks are 
shown in Figure 10. As with all of the pilots, 
Pilot 6 spent the first several minutes scanning 
the traffic. At 6:30, Pilot 6 was alerted to a 
conflict with AC303 with a Level 2 alert (Figure 
10, position A). Note that Pilot 5 received a 
Level 1 alert (traffic advisory with no action 
required) in the strategic mode at this point. The 
tactical mode has no Level 1 alert because intent 
information is not available. At the time of the 
Level 2 alert, Pilot 6 had just started his 
secondary task and decided to complete it before 


resolving the conflict. This indicates that the 
pilot understood he had five minutes until loss of 
separation and did not need to act immediately. 
Forty-five seconds later, the pilot initiated a 
gradual descent to FL310 (position B). He 
maintained his course and continued to scan the 
traffic. At 12:00, the intruder passed overhead, 
clearing own-ship by 1200 ft. Half a minute 
later, Pilot 6 initiated a slow return to his target 
altitude (position C). He was able to easily meet 
the constraints at the RTA waypoint. The pilot 
was also very prompt with performing the 
secondary tasks. 

This case study illustrates how the tactical 
mode appears better suited for blunder scenarios, 
particularly those where the blunder leaves little 
time to react. In both modes, the pilot is alerted 
at the same time to the possibility of loss of 
separation. The tactical-mode pilot is 
immediately instructed to resolve the conflict 
and is given resolution advisories to do so, 
allowing plenty of time to chose and execute a 
maneuver. The strategic-mode pilot is advised, 
however, not to take action prematurely but to 
keep watch on the traffic aircraft. Since 
broadcast intent would presumably be followed 
more often than not in an operational system, the 
strategic approach to reduce unnecessary 
maneuvering while heightening the pilot’s 
awareness of the potential intruder may still 
yield benefits. 

Initial Subjective Data Results 

During the experimental data collection, 
subject pilots were asked to express their 
comments criticisms and suggestions regarding 
the controls, displays and procedures. Two 
questionnaires were used to collect these data: a 
post-simulation questionnaire and a usability 
questionnaire. The post-simulation questionnaire 
queried the pilots on the feasibility of 
maintaining self-separation using two 
operational modes (tactical vs. strategic) and two 
modes of airspace complexity (i.e., traffic 
density, special use airspace, and weather cells). 
The usability questionnaire was designed to 
evaluate the controls and displays in order to 
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identify their acceptability or unacceptability. 
Pilots were asked about the acceptability of the 
altitude and range filters (functions and 
implementation); the display clutter; the 
adequacy of the aircraft data block presentation; 
the use of color coding vs. numeric altitude 
information; the acceptability of the 
climb/descent symbology; and, the usability of 
the RTA (required time of arrival) symbology. 

Post-simulation results 

The post-simulation questionnaire asked the 
pilots to contrast the tactical and strategic 
operational modes from nine operationally 
specific perspectives. These included: flight 

safety, flight efficiency (minimized fuel 
consumption and time to destination), overall 
workload, maintaining situational awareness, 
identifying conflicts, resolving conflicts, alerting 
accuracy (no false alarms), alerting reliability 
(no late alarms or missed alarms), and the 
usefulness of the conflict prevention (no-go) 
bands. The pilots were asked to rate these 
parameters on a scale from 1 to 9, where 1 = 
tactical absolutely better, 5 = tactical same as 
strategic, and 9 = strategic absolutely better. 
The results from the questionnaire are shown in 
Figure 11. 

These results indicate that the strategic 
operational mode was preferred in seven of the 
nine operational categories. In the areas of flight 
safety, flight efficiency, situational awareness, 
and resolving conflicts strategic was rated much 
better (8s) than tactical for the high complexity 
airspace. Flight efficiency was also much better 
for the low complexity operations. Strategic 
was rated better (7s) than tactical for overall 
workload and identifying conflicts for both low 
and high complexity operations. In addition, 
strategic was rated better for low complexity 
operations in the areas of flight safety, 
situational awareness, and identifying/resolving 
conflicts. Finally, strategic was rated better for 
high complexity operations in the usefulness of 
conflict prevention bands. Strategic and tactical 
modes of operation were only rated same in the 
areas of alerting accuracy and alerting reliability. 


The small differences in the individual 
category ratings, and the small differences 
between low and high complexity should not be 
interpreted as necessarily significant results. 
Further processing of the quantitative data is 
required to document the significance. 

In addition to these specific Likert scale 
(subjective rating) results, pilots were also given 
the opportunity to provide expanded written 
comments regarding tactical vs. strategic 
operational modes in the areas of: 

• Flight safety and efficiency 

• Pilot workload and attention 

• Traffic information and conflict- 
management tools 

• Acceptance of the self-separation task 

Four specific questions in these areas were 
asked of the pilots at the end of the post- 
simulation questionnaire after they had 
completed flying 12 scenarios, six each in 
tactical and strategic. Responses from the 16 
pilots have been reviewed, and representative 
quotations from the questionnaires are presented 
below in the form of opinions supporting 
strategic or tactical modes. 

Please compare and/or contrast the tactical 
and strategic modes of operations in terms of 
flight safety and efficiency as traffic density 
and airspace hazard density (i.e., operational 
complexity) were varied. 

Strategic mode: 

“With strategic I was able to look ahead 
farther and more quickly assess the most critical 
target both in terms of time and magnitude of 
flight path changes required. With tactical, I felt 
surprised by conflicts, especially vertical.” 

“Strategic is much better. Having an idea of 
the other aircraft’s intent allowed me to make 
better decisions about resolving conflict and to 
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avoid conflict alerts.” 

Tactical mode: 

“Strategic mode in high density offered too 
much clutter for my comfort level. I felt the 
tactical only was safer and more efficient 
because it was quicker and easier to use and 
required less brain RAM. As traffic density 
decreased, the advantages of one over the other 
decreased.” 

“Tactical maneuvering with heading + 
altitude changes seemed to work best in higher 
complexity situations.” 

Please compare and/or contrast the tactical 
and strategic modes of operation in terms of 
pilot workload and pilot attention to the 
separation assurance task? 

Strategic mode: 

“Strategic is much better - allowed me to 
‘stay ahead’ rather than just react to conflict 
alerts.” 

“Tactical placed a higher workload on me 
due to the fact that I was mentally computing 
closure rates and headings. It was better to let 
the computer figure it out in the strategic mode 
and then verify the route using the rest of the 
tools.” 

Tactical mode: 

“Tactical was less workload due to less info, 
but more stressing to resolve due to time 
element. Pilots like to be in control and know 
what’s coming.” 

“Pilot workload increased when in strategic 
operation - need to pay more attention and plan 
further ahead.” 

Please compare and/or contrast the tactical 
and strategic modes of operations in terms of 
availability, utility, and reliability of traffic 
information and the conflict-management 


automation tools (conflict prevention, conflict 
detection, conflict resolution). 

Strategic mode: 

“Strategic allows more conflict prevention in 
that I could take earlier, smaller state changes, or 
avoid them entirely by knowing other aircraft’s 
intent.” 

“I liked almost everything about strategic- 
mode tools better. I used the expected flight 
path a lot. However, I can see where it can 
potentially lead to complacency (people may 
deviate). Will need to always confirm that 
suggested resolution re-routes avoid weather and 
SUA.” 

Tactical mode: 

“Often times too much information is given, 
i.e., if this guy maybe does this, then you may 
have a conflict. Often works better when it’s in 
black and white. Either you do or you don’t, 
plus no gray area. I found tactical better/easier 
because less information was available.” 

Please compare and/or contrast the tactical 
and strategic modes of operations in terms of 
your acceptance of the expanded 
responsibilities and tasks of the pilot in the 
En Route Free Maneuvering concept that you 
experienced here. 

Strategic mode: 

“Once you used the strategic mode and 
trusted it, the workload dropped. There were 
more opportunities to pick up targets and not 
rely on the ‘brain’ to make the right choices.” 

“Strategic required no decision-making - 
simply hit the accept key and modify the 
airspeed which the FMS should be able to do 

anyway.” 

Tactical mode: 

No pilot comments were received that 
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specifically expressed a positive association 
between the tactical mode and acceptance of the 
self-separation task. 

These observations support several accepted 
notions regarding the viability of the DAG-TM 
concepts and tools. First, the pilot community is 
diverse and it may be difficult initially to 
achieve universal acceptance. Second, the 
subject pilots in the experiment demonstrated a 
wide variety of understanding of the difference 
between tactical and strategic operational 
modes. There were those who trusted and 
accepted the strategic resolution with little or no 
thought to resolving the conflict presented or 
assuring themselves that other conflicts would 
not be caused by accepting the resolution. In 
contrast, there were those who wanted to 
maintain control of the aircraft, treating the 
strategic resolution as an advisory and preferring 
to accept the additional monitoring and 
cognitive workload associated with making the 
heading, altitude or speed changes manually 
through the FCP. Third, it is well documented 
that humans revert to “early learning’’ in 
stressful or high workload conditions. In this 
case, the pilots tended to revert to heading, 
altitude, and speed changes using the FCP, 
which they use currently in every flight, over 
acceptance of an automated conflict resolution 
flight plan with which they have no operational 
experience. 

Usability results 

This questionnaire asked the subject pilots to 
evaluate and comment on specific areas of the 
user interface design. Median results of the 
effectiveness of several design features are 
shown in Figure 12. Except where noted, the 
rating scale was: 1 = completely ineffective; 4 = 
borderline; and 7 = completely effective. Seven 
of the eight design features were rated very 
effective to completely effective. The 
orientation of the 3 -digit altitude tag shown with 
each aircraft symbol received a rating of 5, or 
somewhat better than borderline. The rating was 
likely tempered by some of the pilots who stated 
they preferred all text to be horizontal. 


Additional evaluation areas included the 
acceptability of the display clutter, the use of 
text vs. color coding for altitude information, 
and the desire for state-vector predictors. 
Display clutter was rated above borderline (5 on 
7 point scale). Pilots were often observed to 
control the clutter using the altitude filter, 
turning the filter off for a brief scan of all traffic, 
and then turning it back on for decluttering. 
Pilots were evenly split on the reliance on 
altitude tail tags vs. color coding for traffic- 
aircraft altitude information. Ninety three 
percent of the pilots desired the ability to display 
state-vector predictors on traffic aircraft, and 
eighty percent wanted control over the length of 
the predictors. Overall, no interface issues were 
rated borderline or lower. 

Overall, these subjective results should be 
interpreted as acceptance and/or approval of the 
control/display interface design and the 

information content provided. The small 
differences between individual ratings should 
not be interpreted as statistically significant 
results, but simply as pilot preferences based on 
data collected from 16 pilots at the end of 12 
scenarios and 192 segments of simulated use of 
these tools. Further processing of the 

quantitative data is required to document the 
significance. 

Conclusions 

An experimental investigation was conducted 
to compare two possible operational modes for 
autonomous aircraft in a DAG-TM concept for 
air traffic operations under realistic constraints. 
The tactical mode was characterized primarily 
by the use of state information for conflict 
detection and resolution and an open-loop means 
for the pilot to meet operational constraints. The 
strategic mode involved the combined use of 
state and intent information, provided the pilot 
an additional level of alerting, and allowed a 
closed-loop approach to meeting operational 
constraints. Operational constraints included 
separation assurance, meeting an RTA, avoiding 
weather cells and SUAs, flying efficient 
trajectories, and maintaining passenger comfort. 
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In addition, CDTI design features for 
autonomous operations were evaluated for 
usefulness and effectiveness. 

Preliminary results indicate that pilots in both 
modes were generally able to meet the 
operational constraints. Functional differences 
between the modes were evident in scenario case 
studies. In scenarios with conflicts based only 
on state vectors, pilots operating in the strategic 
mode were less frequently observed to maneuver 
unnecessarily. Under tactical situations, pilots 
sometimes caused several additional conflicts in 
their maneuvering to resolve the initial conflict. 
In scenarios with conflicts based initially on 
intent, strategic-mode pilots generally took 
advantage of the ability to resolve the conflict 
earlier than the tactical mode allowed. In 
blunder scenarios, the lack of intent information 
in the tactical mode generally resulted in 
resolution of the conflict before the blunder 
occurred. 

Subjective data results indicated a consistent 
pilot preference for the strategic mode of 
operations over the tactical mode. However, 
supportive and constmctive statements were 
received for both strategic and tactical modes, 
indicating the following conclusions. The pilot 
community is diverse, and it may be difficult 
initially to achieve universal acceptance of a 
common set of tools and procedures. The subject 
pilots had a wide variety of understanding of the 
difference between tactical and strategic 
operational modes, given the short time 
available for familiarization and for building 
experience and trust. Pilots were generally 
supportive of the CDTI design features 
supporting both operational modes and offered 
many helpful suggestions for further 
improvements. Although the strategic 
operational mode is relatively immature and 
undeveloped relative to the tactical mode, the 
experiment highlighted many potential benefits 
of the strategic mode to aid in meeting realistic 
operational constraints, indicating that further 
development and exploration of the strategic 
mode is warranted. 
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Figure 1. Effective times for maneuver and priority flight rules. 
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Level 1 


Level 2 


Level 3 


Figure 3. Traffic symbology for the alerting levels. 





Figure 4. Conflict types investigated in the current experiment: intent only (I); state only (S); blunder (B). Intruder 
is aircraft approaching from the right. Diagrams are generic and do not represent the specific geometry investigated 

in the experiment. 





Figure 5. Recorded tracks of state-only conflict scenario flown by Pilot 1 in strategic mode. 



Figure 6. Recorded tracks of state-only conflict scenario flown by Pilot 2 in tactical mode. 
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Figure 7. Recorded tracks of intent-only conflict scenario flown by Pilot 3 in strategic mode. 



Figure 8. Recorded tracks of intent-only conflict scenario flown by Pilot 4 in tactical mode. 
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Figure 9. Recorded tracks of blunder conflict scenario flown by Pilot 5 in strategic mode. 



Figure 10. Recorded tracks of blunder conflict scenario flown by Pilot 6 in tactical mode. 
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Figure 12. Pilot ratings of user-interface design feature effectiveness. 
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